Papers with BART model
Hie-BART: Document Summarization with Hierarchical BART (2021.naacl-srw)
Copied to clipboard
| Challenge: | Existing document summarization models do not capture hierarchical structures of documents . proposed model incorporates multi-granularity self-attention (MG-SA) |
| Approach: | They propose a new abstractive document summarization model, hierarchical BART . the proposed model captures hierarchically structured sentences in the BART model . |
| Outcome: | The proposed model outperforms baseline models and improves performance on CNN/Daily Mail dataset. |
PAIR: Planning and Iterative Refinement in Pre-trained Transformers for Long Text Generation (2020.emnlp-main)
Copied to clipboard
| Challenge: | We present a content-controlled text generation framework for pre-trained Transformers . large pre-train models are the cornerstone of many state-of-the-art models in natural language understanding and generation tasks. |
| Approach: | They propose a content-controlled text generation framework that adds content planning to large pre-trained Transformers without modifying model architecture. |
| Outcome: | The proposed framework improves the quality of the outputs on three domains. |
Incorporating Behavioral Hypotheses for Query Generation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Prior work has focused on extending standard Seq2Seq models but literature often leaves out the influence of clickthrough actions. |
| Approach: | They propose a generic encoder-decoder Transformer framework to generate query suggestions from user inputs. |
| Outcome: | The proposed approach improves top-k word error rate and Bert F1 score compared to a recent BART model. |
SGSH: Stimulate Large Language Models with Skeleton Heuristics for Knowledge Base Question Generation (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods have significantly boosted the performance of Knowledge Base Question Generation (KBQG) through pre-trained language models thanks to the richly endowed semantic knowledge. |
| Approach: | They propose a framework to Stimulate GPT-3.5 with Skeleton Heuristics to enhance KBQG by combining skeleton heuristic guidance with a soft prompting approach. |
| Outcome: | The proposed framework incorporates "skeleton heuristics" which provides more fine-grained guidance associated with each input to stimulate LLMs to generate optimal questions. |
Few-Shot Pidgin Text Adaptation via Contrastive Fine-Tuning (2022.coling-1)
Copied to clipboard
| Challenge: | Currently, low resource languages are not supported by proper translation systems or parallel corpus. |
| Approach: | They propose to fine-tune the pretrained language models to generate utterances in English-to-Pidgin by leveraging the proximity of the source and target languages and using positive and negative examples in constrastive training objectives. |
| Outcome: | The proposed method is sufficient to generate utterances in English-to-Pidgin, which are two closely-related languages. |
HealthAlignSumm : Utilizing Alignment for Multimodal Summarization of Code-Mixed Healthcare Dialogues (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Collaboration between doctors and AI scientists is leading to personalized models to stream-line healthcare tasks and improve productivity. |
| Approach: | They propose to use alignment techniques to combine a doctor-patient dialogue with a visual component of the BART model. |
| Outcome: | The proposed model in-tegrates visual components with the BART ar-chitecture. |
Tokenization Consistency Matters for Generative Models on Extractive NLP Tasks (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Pretrained sequence-to-sequence (seq2sequ) models have been widely used to solve extractive tasks, where parts of the input are extracted to form the desired output. |
| Approach: | They propose a simple fix to tokenization inconsistency that damages extractive nature of generative models by causing performance drop and hallucination. |
| Outcome: | The proposed model performs better in both in-domain and out-of-domain datasets with a notable average of +1.7 F1 gain when a BART model is trained on SQuAD and evaluated on 8 QA datasets. |